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ABSTRACT 



The major advantages in computer-assisted instruction 
(CAI) are the potential for the individualization of instruction and 
the savings in instructional time. CAI's major disadvantages include 
the large amount of time required for course development, personnel 
required, initial outlay of funds, and the total cost. Other 
disadvantages, due to the present state of the art, are the small 
amount of software readily available and the short supply of 
personnel able to develop CAI programs. The three main methods of 
programing, in order of complexity, are linear, branching, and 
multi- track . These methods are explained briefly as preface to a case 
study of CAI courses at the U.S. Naval Academy. Procedures for course 
development, course revision, course conduct, and course evaluation 
are reviewed. Recommendations for personnel and time resources, as a 
result of the Naval Academy experience, are set forth, (JK) 
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INTRODUCTION 



Computer assisted instruction is an instructional 
method which uses the computer as an integral 
part of instruction. The computer acts as an in- 
structor, presenting the material, asking questions 
to check comprehension, presenting more material 
and testing. 

Since it is a new instructional and experimental 
method, CAI is being used to a limited extent by 
the Federal Government. As is often the case with 
new instructional methods, trainers tend to over- 
react in one direction or the other; some think OAI 
is the solution to all training and education prob- 
lems. while others think it will never be practical 
and effective and therefore should not be used. 
Neither of these attitudes is valid; the value of 
OAI lies between these extremes. 

Very few training specialists know much about 
OAI: it is therefore difficult for them to make a 
rational decision about the appropriateness of it 
for a particular situation. It is not enough to know 
that OAI worked for someone else, the conditions 
under which it worked must be known. It is not 
enough to say that OAI in general is effective and 
efficient, it is only effective under certain condi- 
tions. Although the ideal conditions for CAI are 
not yet known, several institutions who have used 
CAI have some idea what these conditions are. 
The purpose of this paper is to give a general dis- 
cussion of OAI and to focus on the U.S, Naval 
Academy’s specific experiences in this area. 

In the absence of specific knowledge of the effec- 
tiveness of OAI, what criteria can the Federal 
trainer employ when considering the use of CAI ? 
In this paper, the following points, among others, 
must be considered : the availability of appropri- 
ate resources, the type of subject matter, the im- 
portance of the need for individualization and the 
number of students. This pamphlet is designed to 
provide training specialists with a discussion of 



these considerations so that they can make a more 
meaningful decision in relation to OAI. 

Since PI is an integral part of CAI, an under- 
standing of PI is useful. For those who do not al- 
ready have this knowledge, a review of a previous 
pamphlet in this series, {Programmed Instruc- 
tion: A Brief of its Development and Current 
Status ) , is advisable. 

The following five cautions should be kept in 
xu md as this pamphlet is read : 

1* Don’t act as if computer-based training 
were something entirely new. Always con- 
ceive of computer-based training in such a 
way that you can relate it to what is known 
about conditions for effective training. 

2, Don’t be misled into thinking that a com- 
puter offers a training system which is ob- 
viously so superior that no evaluation need 
be made or records kept, 

3, Don’t just apply the computer to training. 
Carefully analyze the total training process 
and apply the computer to those functions 
that need automation, 

4, Don’t be seduced by claims that, almost 
mysteriously, profound learning and un- 
derstanding occurs when a trainee “uses a 
computer to interact heuristically with the 
subject-matter.” Systematic practice with 
feedback is still the best condition for 
learning, 

5, Don’t put all of your “eggs” in any one type 
of research and development “basket”. Be- 
cause of high investment costs and other 
factors, there seems to be a somewhat un- 
healthy emphasis on an immediate 
capability, 1 



See footnotes at end. 
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Computers may be used in many ways in the 
educational process: in administration, testing, 
record keeping, simulation and computer assisted 
instruction* 

, . . many people use the term “computer-assisted in- 
struction’* ... to cover a wide variety of different con- 
dition s. Careful reading often is required to determine 
whether, by “computer-assisted instruction, ” the author 
means: (1) Administrators using computers to store and 
process administrative data about students, (2) students 
using computers to solve problems, e.g. mathematics, (3) 
students using computers which represent complex rela- 
tionships, e,g, math, physics, chemistry labs, etc,, (4) a 
computer presenting linear-type programmed instruction, 
or (5) computers in highly individualized and interactive 
tutorial instruction. 

Frequently, the involvement of a computer in the in- 
structional process seems to blind one to many of the more 
significant functional properties of that instructional 
system, Those functional properties are the more impor- 
tant characteristics, not that a computer is ^r is not 
involved. 5 

The first four of these functions use computers’ 
solely to facilitate the eductional process; while 
this is a use of computers in eduction, it is not a 
use of computers to educate. The last category, 
computer assisted instruction or CAT, uses the 
computer as an intrinsic part of the learning proc- 
ess. In OAI the computer is used for individualiz- 
ing instruction* 

Computer assisted instruction (CAT) is a rela- 
tively new instructional method which uses the 
computer in conjunction with programmed in- 
struction (PI). As is true of any other in- 
structional method, CAT is not a cure-all for 
instructional problems ; it must be used only when 
it is appropriate. 

For some of us, it has become illogical to attempt to solve 
problems In educational strategy by oms medium, alone, 
whether it be film, television, or language laboratories, 
although some of these media are almost s,v sterns in them- 
selves. The systems view of media hold^ tfcot the digital 
computer, no matter what its performance characteristics 
may be, is just another important component of a larger 
assemblage of instrumentation needed to implement 
favorable conditions for learning. 11 



Sf e footnotes at end. 



Like any other instructional method it has its ad- 
vantages and disadvantages, is appropriate for 
some situations and not others. Since CAI is based 
on PI, it has some of the same advantages and dis- 
advantages. u . . . OAI in its most primative sense 
is merely PI with a sort of electronic page 
turning.” 4 

PI is an instructional method in “which the student is 
lead through a series of questions, responses, and confirma- 
tion of his responses until little by little he has progressed 
from small bits of relatively simple knowledge to more 
complex principles. 

Many writers have noted the similarity between the PI 
approach and the tutorial system which asks selective 
probing questions of a student, aud then lets him do the 
work of learning. The tutor must, know his objective and 
dissect the subject matter into small bits of information. 
Through a logical progression of elementary questions and 
answers and through reward or correction of answers, the. 
tutor brings the students to a grasp of the whole concept. 
Thus the procedure may be likened to the building of a pic- 
ture puzzle where the complete configuration comes only 
through the correct placement of individual parts.® 

The computer can be programmed to adapt the 
learning situation to each individual’s learning 
difficulties which would burden an instructor with 
many trainees. Thus a student can progress at his 
own rate. The computer allows for true individ- 
ualization of learning. In some cases the computer 
presents the learning material to the student by 
means of a cathode-ray tube (CRT), a television- 
like screen. A question is then asked of the student. 
If the trainee responds correctly, he is directed to 
the next bit of information. If he answers incor- 
rectly, he is directed to a bit of material which is 
specifically designed to remedy the particular type 
of mistake which the student made. If the student 
still does not learn, he is again given additional in- 
formation. The student then is presented the next 
bit of information. This whole procedure is done 
automatically 5 the student is not required to flip 
pages or turn the book upside down to check his 
answer. The computer can also provide individ- 
ualized homework assignments based on specific 
deficiencies. 
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Advantages and Dif advantages of 
CAI Over Other Instructional 
Methods 

There are certain advantages to using CAI 
rather than some other instructional method. The 
major ones are the potential for individualization 
of instruction and the savings in instructional time. 
The major disadvantages of CAI include the large 
amount of time required for course development, 
personnel required, initial outlay of funds and the 
total costs. 

Advantages of CAI 

Individualization of Instruction 

CAI has practically unlimited possibilities for 
individualization of instruction, while PI has the 
potential for a very limited amount of individuali- 
zation of instruction. 

The true individualization of instruction is pogsihA and 
represents the real departure of CAI from programmed in- 
struction, Thus, while programmed instruction provided 
the initial break in thinking about individualized instruct 
tion and some useful functions, GAI provides the means 
by which we can put that thinking to work, and add new 
functions, the problem is how to adjust the fabric of in- 
struction to the form of the students* aptitudes, a per- 
sonality and beginning knowledge— the texture of his entry 
behavior. Basically we are dealing with a problem of 
adaptive instruction. By that we mean that the instruction 
must depend on knowledge of the student.* 

Although the computer is limited in that it can 
store only a certain amount of information, this is 
generally adequate. The only real limitation is 
the ability of the author. 

In PI, the next bit of information which is given 
a student is determined only by the answer he gave 
to the preceding question. In CAI, the next infor- 
mation given depends not only on the responses 
made to the previous question, but on either all or 
many previous responses. If .the student had diffi- 
culty with a certain type of learning process, re- 
gardless of subject matter, a good computer pro- 
gram ib capable of emphasizing the modules in 
which the process occurs, as well as reinforcing 
individual modules by providing additional 
materials. 



Savings in Instructional Time 

Another major advantage of CAI is that is 
generally requires less time than traditional meth- 
ods to teach the same amount of material, “One 
of the most consistent findings with CAI tutorial 
applications is marked savings in instructional 
time with no loss in post- instructional achievement 
performance, 57 7 The TJ.S. Army Signal Center and 
School reports the following in their feasibility 
study of CAI : 

• Based on the criterion measure there was no sig- 
nificant difference between instructional methods. 
Significant differences were found in performance 
among students of different aptitude levels, 

• The mean time required to complete the GAI course 
was about 11 percent less time than the teed time 
for instructor-controlled and television-con trolled 
instruction when all aptitude levels were included. 
High level students averaged 49 percent faster, 
medium level students averaged 17 percent faster, 
and low-level students averaged 32 percent slower 
than those taught by conventional methods,® 

One year later, a similar study was conducted 
by USASC,® and it was found that, due to course 
revisions, the mean amount of time to complete the 
CAI course was decreased by an additional 9 per- 
cent, for a total of 20 percent less time to complete 
a CAI course than the teacher- administered course 
(TAI), 

Human Resources Research Organization 
(HumEEO), which is developing a model CAI 
system, states the following about the efficiency of 
CAI versus TAI : 

Thus, the most plausible assumption would seem to be 
that, at worst, PI and/or GAI will be no better than the 
average textbook or the average lecture notes taken from 
the average lecturer. We might next ask how conservative 
an assumption of equal effectiveness might be. In assess- 
ing the degree of conservatism It will be well to recall 
the analogy proposed at the outset of this paper. Before 
1910 the airplane and the horse drawn buggy might have 
been equated in their effectiveness as transportation. Up 
to perhaps 1950 equal effectiveness debates might have 
been carried on about airlines and railroads. Today the 
matter is clearly beyond debate. Since GAI is still in a very 
early stage of development, it is reasonable to expect ever- 
increasing effectiveness with its continuing evolution. 

The point is well-illustrated by two studies. In 
the first study, [“Fixed Sequence Versus Branch- 
ing Auto-Instructional Methods,” J. Educ , Pay - 



chat, vol- 52, 1961, pp. 1G6-1T2 I Silberman et al . 
were unable to demonstrate advantages for a 
branching presentation via CAI where the crite- 
rion for branching was error rate. In the second 
study, [“Effects of Branching in Computer Con- 
trolled Auto-Instructional Device , 35 /. Appl. Psy- 
chol vol. 46, 1962, pp. 389-392] Coulson et al. 
were able to demonstrate these advantages after 
revision and refinement of the instructional pro- 
gram and decision criteria were effected. “In the 
accumulation of many small improvements and 
sharpening of techniques, the inherent potential 
of CAI is likely to bo realized in time. Thus, the 
assumption of equal effectiveness is a conservative 
one which almost certainly will become more so 
as time progresses . 55 10 

Other Advantages of CAI are secondary and 
mainly a matter of convenience. The four which 
will be discussed are: safety and expediency of 
instruction, record keephig ability, increased in- 
structor effectiveness, and increased quality of 
training* The individualization of learning itself 
could, under the right circumstances, justify CAI, 
although the other uses of the computer help make 
CAI more cost-effective than the individualization 
of instruction alone* 

The computer can be very useful for dangerous, 
lengthy, or expensive- failure learning situations. 
Laboratory experiments can be very dangerous, 
but with the computer, the dangers of explosions, 
fires, et cetera, are eliminated. Thus the student 
can still check color, temperature, and chemical re- 
actions, but without any of the danger usually 
involved. The time required for many experiments 
in biology can be greatly shortened with the use 
of the computer. Men learning to fly use a Link 
Trainer, a simple computer which simulates all the 
important aspects of instrument flying. 

The computer has the capabilities of keeping 
an automatic record of all student responses. In 
this way, not only can individual trainee perform- 
ance be noted, but the strengths and weaknesses of 
the course can be noted. The trainees also have the 
option of commenting on any question which they 
do not understand, or an answer with which they 
do not agree. If many of these comments are made 
on any particular question, either the question or 



the module need revision. This is an improvement 
over traditional instruction since learning difficul- 
ties can be pinpointed to a particular module. This 
facilitates revision since incorrect answers can be 
directly associated with a particular unlearned 
module. 

CAI does not eliminate the need for instructors, 
it merely puts the emphasis of an instructor’s role 
in a different light. The instructor has fewer tra- 
ditional demands on his time since he has a lighter 
teaching load and uses less time to evaluate trainee 
learning. He does not need to spend time main- 
taining records* As a result, he has more time to 
spend with the unique problems of each individual 
Since the instructor is relieved of the tedious work, 
his time can be more creatively and effectively 
spent. 

The quality of training can be increased, since 
it is possible through CAI for all students to have 
an equally good instructor. In traditional learning, 
some instructors are better than others, while in 
CAI, the best instructor can be chosen. Since so 
many trainees can use one course presented by the 
same instructor, it is important that he be effective. 
In traditional classrooms, the influence of any one 
instructor is limited, while in CAI it can be ex- 
tremely extensive. 

Disadvantages of CAI 

The disadvantages of CAI can be put into two 
categories, those which are a result of the state of 
the art of CAI, and those inherent in CAI itself, 

State of the Art Disadvantages 

Software 

The former includes mainly the software portion 
of the OAI system* Unfortunately, problems sur- 
rounding software are often overlooked when CAI 
is being considered as an instructional method. 
Very few programs are available for off-the-shelf 
purchase, which is not usually mentioned by hard- 
ware salesmen. Two authors stress this in their 
discussions of CAI; 

The press implies that the heavy investments of Wall 
Street and of giant new industries have rounded up the 
brains of the nation, have produced quality programs in 
quantity, and are ready to move with dispatch into the 
schools. It was a real shock to discover how crude and 
primitive the programs actually are, and how far they 
are from the large-scale integration into the schools or 
into the educative process in general . 31 
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In the fleld of computer design the most severe lack of 
knowledge is not how to design and build bigger and faster 
machines, but how to mike them function, how to integrate 
them into the human world, and how to make them do 
what we want them to do. Norbert Wiener’s later writing 
harped upon the dange:* we risk by building machines to 
perform functions that we do not adequately understand. 
The dangers are real because of our ability to understand 
the purpose to which they might be put; and we could 
end by putting electronic machines to uses we u ould not 
want to put them to if we really understood what the uses 
were , 12 

It is known by people who work in the field that 
CAI is not approximate for all types of subject 
matters; unfortunately which type of subject mat- 
ters are moLt appropriate is not known. This was 
not always considered when existing courses were 
developed. Some of the CAI courses which have 
been developed would have been more effective 
with some other instructional method, This, in 
addition to the small amount of software available, 
greatly limits the number of effective CAI Tiro- 
grams available. 

Extensive research still needs to address this question. 
We believe, however, that at the present state of the art 
of producing OAI materials, certain types of strategies, 
and obviously that subject matter which is seemingly 
dependent on those strategies lor effective presentation, 
are overly ambitious. Thus, our experience* would lead 
us to avoid the initial presentation of concepts (total 
tutorial approach) for college level subject matter of 
some complexity. Similarly, we would avoid relating the 
system to mechanical page turning employment, whereby 
large {.mounts of text are presented without any attempt 
at periodic evaluation of individual progress. Well struc- 
tured, tutorial exposition is reasonably undertaken for 
upper level course materials. Beview, drill, simulation, 
gaming, and evaluation for most disciplines are also 
within the realm of economic attainment . 12 

Persormel 

Since software is generally not available for 
purchase, it must be developed in-house or by a 
contractor. Individuals possessing this skill are 
seldom available in-house to n - Federal agency. “It 
is unlikely that classroom teachers or school sys- 
tems will be more able to prepare computer pro- 
grams in the future than they have been able to 
write textbooks in the past.” 14 This statement ap- 
plies to Federal trainers as well. No one individual 
combines all the capabilities necessary for the de- 
velopment of programs; a team of individuals is 
required consisting of ; author, instructional pro- 



grammer, audiovisual expert and behavioral scien- 
tist, among others. The U.3, Army Signal Center 
and School lists certain individuals as those neces- 
sary as the basis of a US AOS C CAI project team 
(see fig, l). 1 5 The same personnel would be re- 
quired for any CAI project, but the numbers of 
some of the types of individuals may vary depend- 
ing on the number of courses in CAT being used. 
It is obvious that no one person can fill all these 
roles. Often these different types of individuals 
are not on the staff and must be recruited on a full 
or part-time basis. 

Inherent Disadvantages 

The inherent disadvantages of CAI cover such 
things as cost, time, initial outlay as well rp other 
practical considerations. 

Cost 

Although the cost of OAI is very high when 
compared with other instructional methods, OAI 
can be made cost-effective. The cost of a CAI 
course is much greater than a course developed 
in another instructional method. OAI course de- 
velopment requires several times as much time as 
TAI courses. Most Federal training courses use 
little hardware; thus the cost of the hardware in 
CAI is much greater than that in TAI. 

The different costs were broken down by the 
Continental Army Command in a feasibility study 
of CAT.™ There are two general categories of 
costs, OAI systems costa and program development 
costs. 

I. CAI Systems Costs 

A, Capital development costs— this includes 
the hardware, installation and facilities. 

B. Continuing costs— this category includes 
those costs which require periodic outlay 
of funds such as; system maintenance; 
operations, such as cooling, humidity con- 
trol, and electricity; program adaptation 
and maintenance — which requires person- 
nel and materials to update and modify 
courses ; overhe ■ and supplies, 

II, Program Development Costs 

A. Course material preparation. 

B. Course implementation and debugging 
(categories A and B include the salaries 
of programmers, coders, analysts) , 
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0. Training aids. 

D. Overhead. 

CAI courses differ from other instructional 
methods in that a large amount of money is re- 
quired initially to purchase the hardware. To 
other types of courses, audiovisual equipment can 
be obtained piece-by-pieee as funds allow, while 
in CAI courses, a large amount of money must 



be spent even before the course is developed. This 
would be difficult for agencies who get. training 
funds on a yearly basis and would have to spend 
all 1 year’s training money on a CAI system in 
the first few months of the year. 

There are two seemingly conflicting opinions 
about the cost competitiveness of CAI : “So, even- 
tually, the oost/benefits issue must be faced. The 
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comparison is inevitable. And for the moment, 
at the public school level, that comparison cannot 
help but be unfavorable for CAL” 17 Another 
opinion asserts that: “This illustrates the general 
principle that savings in instructional time ob- 
tainable with OAI will more than offset the addi- 
tional costs of the CAT equipment and lead to 
a net reduction in total training costs per course 
graduate,” 1S It is true that many CAI courses 
are not presently cost-effective; however, as 
trainers become more Imowledgeabl© in this in- 
structional method, the cost of CAT should eorm 
pete favorably with TAX. The second quotation, 
above, refers to a purchased computer system used 
for 18 instructional hours a day, 5 days a week, 
with 24 student terminals. With fewer terminals 
or instructional hours on the computer, a CAI 
system would not be cost-effective. The important 
factor is the amount of time the computer is used. 
Unless it is used to the maximum it will not be 
efficient. During the developmental stages of a 
system, the stress is put on making it effective. 
Later, when it has been proven effective, the effi- 
ciency as well as other practical aspects are im- 
proved. CAI is still in the developmental stages 
and so is not as efficient as it will be in the future. 
Although it is difficult to make CAI cost-effective 
under present conditions, in the future this may 
change, “Given the above set of assumptions along 
with the reasonable notion of at least equal effec- 
tiveness for CAT and TAI, it is predicted that, 
less than 10 years from now, CAT costs — including 
the development expense of a number of (600) 
courses— will be roughly half the cost of com- 
parable TAI,” 10 

It is difficult to predict how much the software 
development will cost, but the hardware costs are 
quite standard. 



Lret*s consider what experience to date has shown to be 
the requirements of large scale CAI centers for annual 
funding levels. From informal discussions with knowl- 
edgeable persons regarding their CAI projects’ fiscal 
problems, it appears both internationally and in the 
United States that an operating budget of approximately 
$250,000 to $300,000 a year is necessary merely to main- 
tain facilities in operation. The reason for this becomes 
quite clear if one considers simply the rental price of 
an IBM 1500 system as an example. The hardware alone 
averages $100,000 to $110,000 per year. 50 

Time - 

CAI courses represent an enormous investment 
of time. The U.S. Naval Academy reports using 
between 45 and 350 hours of author time per CAI 
instructional hour and between 120-625 hours of 
programming time per CAI hour. Different vari- 
ables affect the amount of time necessary to de- 
velop and program a course. 

Since CAI is a new field, it takes more time at 
present to develop a CAI course than it will in 
the future. Each author will not have to learn for 
himself how to avoid lengthy mistakes since guide= 
lines will be available to him. But because of the 
nature of OAI, course development will always 
be considerably more lengthy than traditional 
course development. The time used for course de- 
velopment will always be a disadvantage, but will 
become leas so with experience. 

There are other inherent disadvantages of OAT 
which apply particularly to Federal government 
and industry rather than the school system: (1) 
CAI is only practical if a large number of trainees 
will be using the course; it is not economical to 
develop for only a few; (2) the subject matter 
must be of a relatively stable nature or it will not 
be economical because of the changes which will 
be constantly necessary. 
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THE LINEAR PROGRAM 31 




Programming Methods 

The three main methods of programming, 
whether for PI or CAI, and in order of complexity, 
are linear, branching and multi-track. In the lin ea r 
method, the trainee is presented with a bit of mate- 
rial and is asked a question about it. After he 
answers, he is told whether or not his response is 



correct, he then proceeds to the next bit of 
information. 

The branching program is similar to the linear 
except at the point when the trainee gives his 
response. If the response is correct, he moves on 
to the next bit of information. If it is incorrect, 
he is given some remedial instruction designed to 
correct his error. A different type of remedial in- 
struction is designed for each incorrect response. 



THE BRANCHING PROGRAM 22 




THE MULTI-TRACK PROGRAM 23 



The Multi-Track Program other types as can be seen by the following 

This type of program is different than the two diagram. 




Several different versions of the same course are 
developed for different levels of trainees. A pre= 
test is given and the trainees are placed according 
to the results. The same material is presented but 
with varying step sizes. “A superior student will 
be able to use a relatively short program with 
fairly large steps enabling him to remain inter- 
ested in the material A poorer student will be led 
through the same information in a longer pro- 
gram with very small steps in order to minimize 
his misunderstanding or -getting lost’ in the pro- 
gram.” 24 This type of program would be very 
cumbersome if it were in text form (or on a teach- 
ing machine) while the computer can handle it 
easily. The branching and multi-trneic programs 
take advantage of the computer’s capabilities 



for the development of highly individualized 
instruction. 

The most effective use of the computer, how- 
ever, is when multi-track is combined with branch- 
ing. In this case, the student works at his own 
level: lie is given remedial instruction according 
to the type of error made when he responds to a 
question. Additionally, with the computer, the 
type remediation and the next bit of information 
given is determined by the student’s total previous 
responses. 

So far a general introduction to OAI has been 
presented; some advantages and disadvantages 
have been outlined. The next section will describe 
a case study of an actual application from its de- 
velopment to its current status. 
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CM AT THE ILS. NAVAL ACADEMY 



“The amount of Imowledge needed by junior 
naval officers, like many of their civilian peers, 
has increased exponentially in the past decade. 
Conventional teaching methods have been tre- 
mendously pressed to meet this need. To find some 
means of relieving this pressure, the U.S. Naval 
Academy and other Navy schools have tried such 
‘non-eon ventionaP techniques as programmed 
texts, language laboratories^ educational closed- 
circuit television, and the like. An emerging tech- 
nique— computer- assisted instruction, or CAI — 
was made the subbject of a two-part investigation 
to determine its effectiveness in midshipman edu- 
cation,” The first part, the CAI-Teletype Proj- 
ect, utilized teletype terminals to access a remote, 
commercial computer system. The second part, the 
CAI-1500 Project which is discussed here, utilized 
a dedicated instructional computer system (the 
IBM 1500/1800; with 25 remote, multi-media stu- 
dent terminals. 

The application of OAI in other institutions 
was studied to determine its effectiveness. From 
the findings obtained, it seemed that the Naval 
Academy could alleviate the above problem 
through the use of: OAI, The following capabil- 
ities of CAI can make instruction more effective : 
“The computer offers a ‘management sys- 
tem’ that can incorporate all instructional 
media — such as programmed texts, films, tele- 
vision, and simulators, 

“Students will receive individualized in- 
struction. 

“Courses can be taught in greater depth 
“Courses could contain more material if 
taught in the same length of time as tradi- 
tional courses, or 

“Length of individual course could be 
shortened and time used for other military 
training, 

“Subject matter will be effectively presented 
via the most appropriate instructional media, 
“Course designers will be aided in deter- 
mining optimum course content.” 

Course Description 

The Naval Academy began its extensive con- 
tact with CAI | in the fall of 1966, with seminars 



on the subject by the Office of Naval Research and 
General Learning Owp. The Academic Computing 
Center was responsible initially for coordination 
of the activities and plans for the CAI-1500 Proj- 
ect begun in the spring of 1967 The instructional 
computer system was installed that sunn when 
the writing and the programming began. Dur- 
ing the academic year 1967—68 the courses were 
developed and revised. After the completion of the 
validation phase, which began during the academic 
year 1968-69, the evaluation phase was begun. 
Evaluation of the courses has continued into 1971 
and is currently in the final steps, (In mid-1909, 
the group handling CAI and multi-media proj- 
ects was taken from the Academic Computing 
Center and formed into a separate organization, 
designated, first, the Educational and Manage- 
ment Systems Center then, ultimately, the Educa- 
tional Systems Center.) 

The O AX-1500 project began with four courses, 
two of which were subsequently replaced by two 
others. The courses presently being run. Modern 
Physics, General Chemistry, Russian, and Naval 
Operations analysis, use OAI in different amounts. 
None of them use it for the total instruction, the 
total amount of CAI instruction varying from 12 
percent to 83 percent per course. The scheduling 
of the use also varies from course to course, some 
being at a regularly scheduled time, and one being 
scheduled according to the subject matter and 
need. Each course uses OAI in a different way— 
Russian for drills, listening, and reading com- 
prehension; Chemistry for simulated lab exer- 
cises, and problem solving;* and Naval Operations 
for mock tactical exercises and problem solving. 

Course Strategy 

Each portion of the course which uses CAI is 
organized in a somewhat similar way, although, 
overall, a different decision model is now used in 
each course, particularly Russian (fig, 2). The 
student is given a pre-test on each portion of ma- 
terial, or module. If he does not achieve the re- 
quired level for that objective, he must go through 
the subject matter; if he does, he has the option 

•First semester Chemistry is now total tutorial for Gas 
Laws. 
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of going on to the nest module or of going through 
this one. After going through the materials the 
student is tested again to see whether or not he 
learned the appropriate material. If he did not, 
remediation is available ; if he did, he goes through 
the module summary and then on to the next 
module. There are additional aspects and alterna- 
tives to this strategy in some courses, but basically 
it is the same in all. (It is to be noted that this is 
the course strategy, not the programming method 
used.) 

The programming method used most prevalently 
is branching. In this type of programming, the stu- 
dent is presented with a piece of information. 
He is asked a question and the next bit of infor- 
mation presented to him depends on the correct- 
ness of his response. If it is correct, it is reinforced ; 
if it is incorrect, he is given remediation and then 
returned to the question. 

Within a module, the student learns a portion of 
material and then is asked a question on it; if he 
can answer correctly he goes on to the next por- 
tion within the module. If not, he is branched to 
one of many frames (depending on how he 
answered the question) where ha is given the ap- 
propriate remedial learning and then a check 
test. Then he proceeds to the next portion of 
material. 

Course Development 

Course proposals were submitted by each aca- 
demic department which was to have a OAI 
course, A specific format was required which in- 
cluded the following : the scope of the course, ob- 
jectives, project plan, course description and 
faculty participation. 

Seminars on OAI course development were 
offered in the summer of 1966 but not all faculty 
members were able to attend. The courses were 
developed mainly by existing faculty members, 
several working on each course. Very few restric- 
tions were placed on the faculty in developing the 
courses. Each developed his own objectives which 
may or may not have been approved under ideal 
conditions, but due to lack of time had to be ac- 
cepted. Each constructed his own finals and mid- 
terms instead of using the ones already in use in 
non-C A I courses. Later in the program, a course 
development model was established based on ex- 
perience gained from earlier course development. 



Course Revision 

It was assumed that course revision would be 
necessary since CAI was new to most of the in- 
structors. There are two types of revision which 
are necessary in CAI courses. One type is revision 
of a particular course or module (formative) ; the 
other is revision as a result of experience gained 
on the hatch are. 

More extensive revision of the course material 
than usual was necessary since no course trial was 
possible. This means that it was not possible to 
eliminate many of the problem areas before the 
00111*86 was actually given, The input for this re- 
vision came from student performance records 
stored in the computer. When all student records 
are compiled, it ia easy to see which questions and 
modules were ambiguous or not sufficiently cov- 
ered. Revision was also necessary to accommodate 
the different types of learners, which is essential, 
especially in branching programs. Figure 3 ST lists 
some examples why course revisions were 
necessary. 

The other type of revision constitutes “adjust- 
ments’ 5 to the system, that is, learning how to best 
use the equipment. These revisions affect all the 
programs since they are hardware related. For 
example, it was learned that only a limited amount 
of information could be presented on the cathode 
ray tube at one time or it would all be illegible. 
Also, all but the essential graphics and films had 
to be discontinued due to the lengthy preparation 
time of each. 

Although the revisions themselves were rela- 
tively easy, problems were encountered in the com- 
munication between author and programmer. Since 
courses are so complex, it was often difficult for 
the author to pinpoint which particular segment 
was to be changed. Forms were developed later 
in the progarm to alleviate this problem but it 
continued regardless. 

Figure 3 

REASONS FOR COURSE REVISIONS 

1. Typographical or programming error. 

2. Results were poor on this item in the Student 

Response Matrix. 

3. Too many UN’s (unanticipated responses), on 

this item. 
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4, Help i-r ^nested by several students on this item, 

5, Add re iew or back-up capability, 

6, Additional feedback and/or direction needed 
as indicated by poor results on the Student Re- 
sponse Matrix. 

7, Much El (extra instruction) given on this item. 

8, Other 

A number of summer 1970 revisions were 
made on pro and post-test items which were 
not up to standards and were at appropri- 
ate difficulty levels. 

Courses which were developed later in the pro- 
gram required less revision than earlier ones. This 
was partially due to experience with hardware, but 
also to experience in program design and writing 

Course Conduct 

Students are scheduled for the CAI portion of 
the course just as they are for the non-CAI por- 
tions, The students are scheduled for 1 hour at a 
time and are not permitted to leave before the end 
(except in the ease of the Naval Operations Analy- 
sis course, which allows midshipmen to leave at 
any point in a two-period session that they have 
completed the problem) , Some additional time is 
available in the evening for those who need it. 

The students were given a 15 -minute orientation 
on CAX which was found to be the appropriate 
amount of time after various trials. A proctor re- 
mains in the room to take care of any mechanical 
problems which arise. 

The author of the course, or an instructor, stayed 
in the room at the beginning of the program, but 
it was found that the students relied on the instruc- 
tor too much. Now, the instructor sits in a room 
off the CAI room and is able to prepare lectures. 
The Naval Academy found this to be a very effec- 
tive use of both the instructor’s and student’s time. 

Course Evaluation 

The course evalution effort has been divided by 
the Naval Academy into three portions: (1) the 
validation phase, or formative analysis; (2) the 
evaluation phase, or summative analysis; and (B) 
individual differences analysis. The first consists 
of determining whether or not the materials help 
the students achieve the objectives. The second con- 
mats of determining how the effectiveness of CAX 



presentations compares with non-CAI presenta- 
tions. The third consists of determining how the 
effectiveness of the course is affected by individual 
characteristics. This is done separately for each 
course. Each phase will be discussed separately, 

Validation Phase 

The validation phase consists of what is usually 
called testing— testing the students to see whether 
or not they have learned the required material. 
There is a difference however, in that the Naval 
Academy CAI courses use criterion reference test- 
ing instead of the standard normative testing. 
Usually the scores each student obtains on a test 
are compared with all the others, and the grade 
lie receives is based on his relative position within 
the class* The Naval Academy project leaders con- 
tend that each student’s score should be compared 
with the objectives of the course and his grade 
determined by how many of these he achieved. It 
should be noted, however, that this cannot be done 
if the objectives are not clearly defined. 

The validity of the course, this is, whether or not 
the course accomplished what it set out to do, is de- 
termined by the compilation of the student scores* 
The course requires major revision if a predeter- 
mined percent of the students taking the course did 
not achieve a certain percentage of the criterion 
tests, in this case 80 percent by 80 percent. Thj 
revision continues until this standard has been met. 

Preliminary validation began in the spring of 
1969. Due to lack of programmers and course de- 
velopment problems, it was not possible to com- 
plete this phase until spring 1970. 

The tests which were used for the CAI courses 
were generally new ones developed for CAT, This 
means that they had not been checked for validity 
or reliability. This had to be done during the vali- 
dation phase. 

An item analysis was done on all items in the 
test. This consists of determining how difficult each 
item is and how well it discriminates between the 
better students and those who got lower over-all 
scores. By following this procedure, the quality of 
the tests can be assured. 

Evaluation Phase 

“The question typically raised concerning a new 
technique is : How does the new compare with the 
old?” 28 This is the area which is covered in the 



summative analysis. The evaluation design used 
is the experimental, or classical, evaluation using 
control groups and variables. 

The achievement of the group using OAX was 
compared with the achievement of a similar group 
not using CAT. The groups were controlled on 
variables such as SAT scores, grade point aver- 
age and advanced exam scores. By controlling these 
and other variables, the difference in outcome be- 
tween the CAX group and the non-OAI group can 
be directly attributed to the difference in instruc- 
tional method. 

The final evaluation will not be completed until 
the academic year 1970-71. 

individual Differences Phase 

This phase concerns itself with differences in the 
individual courses, students, and instructors, and 
how these differences affect the outcome of the 
course. More specifically, the analyses include : 
Module effectiveness 
Student attitude 
Student correlation 
Predictive equations 

Module effectiveness 

It is necessary to know how effective the indi- 
vidual modules are. This is done by ranking them 
in terms of the amount of possible gain. They are 
then ranked in terms of the number of correct an- 
swers on the mid-term and final exams. These rank 
orders are then correlated. This procedure enables 
the instructor to determine which modules are less 
effective and therefore need more review. 

Student attitude 

The change in student attitude over time to- 
wards CAX materials was measured. Each module 
was rated by students in terms of its perceived 
effectiveness. Also the student’s attitude, or per- 
ceived effectiveness of the modules, was compared 
with the achievement on the modules. 

Student correlation matrim 

Bo the students follow similar paths when com- 
pleting the CAI materials f A student correction 
matrix was generated and analyzed to determine 
whether or not there are any paths which were 
significantly different at the ,05 level. 



Predictive equation 

Some students learn better using CAI than 
others; it would be very useful to know which 
types of students benefit most from CAI. Each 
of the following characteristics was correlated 
with the final grade for each student: SAT- 
Verbal, SAT-Math, English Composition Inven- 
tory, Math-Intermediate, Rank in Class, recom- 
mendations and activities and others. The appro- 
priate amount of each of these variables was deter- 
mined in this way, and an equation siunmarizing 
the right combination can be developed. The equa- 
tion will help in the prediction of successful 
achievement in CAI, 

Results and Cost-Effectiveness 

At this time, the course evaluation ha a not yet 
been completed, therefore no information on 
results is available. When comparing the effective- 
ness of instructional methods, cost muJ. be con- 
sidered as well as the different amounts of gain 
in achievement. Cost will not be available, how- 
ever, until the end of the 5-year program. Until 
then, it is difficult to compare CAI with other 
instructional methods, 

Resources 

Personnel 

The Naval Academy recommends the use of a 
team in the development of CAI courses. Devel- 
oping a program requires several types of skills, 
all of which are not usually combined in one indi- 
vidual. It requires the ability of a subject matter 
specialist, an instructional programmer, an audio- 
visual specialist, and a behavioral scientist. Also 
required are programmers, systems engineers, and 
other technical support personnel. The Naval 
Academy did not have all these skills, so outside 
help was required. 

Authors 

Each of the four courses has several authors, or 
subject matter specialists, who were chosen by the 
departments involved. None of these had previous 
OAX experience ; therefore it was necessary to give 
them some instruction in this area. The amount of 
teaching experience varied from teacher to teacher 
and between CAI courses. 
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The authors ? teacliing load was to have been 
cut back to allow them time to spend developing 
the program. Although this was the goal, this was 
not always possible due to the burden it put on the 
other instructors 3 time. Instructors have duties ad- 
ditional to teaching, which also cut into course 
development time. If the authors had not given 
some of their own time, the time which was allotted 
would not have been enough to develop the courses. 
As a result, the material would have been com- 
pleted much later. The Educational Systems Cen- 
ter estimates that authors spent more off-duty 
hours than work time on course development. 

In addition to course development, some of the 
authors did their own programming, Although 
this was more efficient, because of the author- 
programmer commtinications problem, this fur- 
ther reduced the time available for course 
devel opment . 

Programmers 

Computer programmers were hired and immedi- 
ately needed training on the system. The training 
available to them was inadequate, which resulted 
in programming delays. It was found that on-the- 
job training was more successful than formal 
training so this was given to subsequently hired 
programmers. The system has four full-time pro- 
grammers and from three to 10 part-time. The 
Naval Academy found it difficult to have the ap- 
propriate number of programmers available at all 
times. Although the programmers had slack pe- 
riods, they often had to work overtime, especially 
during certain peak periods. 

Others 

Various other types of personnel were required 
to maintain the system - some full-time and some 
part-time. An art staff is necessary to develop 
graphics and visuals ; mechanical support staff and 
administrative personnel were necessary. Part- 
time assistance was required in the areas of pro- 
gramming techniques and documentation as well as 
validation and evaluation. Several system engi- 
neers were also required. 

Time 

An enormous amount of time is required to com- 
plete a CAT course. The time mentioned here is 



only an approximation but will give some idea of 
how much time to allow. 

Author preparrtur/i time 

Author preparation time has varied in keeping with 
such factors as (1) author knowledge of subject matter, 
(2) author teaching experience, (3) strategy employed 
the CaI materials, and (4) subject matter considera- 
tions that include level of presentation and degree of con- 
ceptualization involved. Acec rdingly, the Academy has 
experienced a wide diversity in production time ranging 
from approximately 350 hours of author time for 1 hour 
of tutorial material in an upper division science course 
to approximately 45 hours for the evaluation and review 
design for 1 hour of Naval Operations Analysis. Current 
revision efforts in General Chemistry show that near re- 
creation of 1 hour* s worth of material requires upwards 
of 150 hours for all authoring activity. 29 

This means that the instruction must be prepared 
well ahead of time, and it is not possible for an 
instructor to prepare a lesson the day before, as 
is now possible in the traditional lecture method. 
It was also found that 

live 2=hour blocks of time, for example, are of much less 
value than two 4-hour blocks. This is because large blocks 
of time are required to ^ up, analyse all the points in- 
volved in teaching the concept, deciding the best way to 
present the various parts, and then generating the ma- 
terials while all this is fresh in their minds. 30 

If the author is to have duties other than C AI 
development, they should be scheduled around this 
work to allow long blocks of uninterrupted devel- 
opment time. 

Program/in/mg Thne 

Programming time is the time which is required 
to put the course into the computer. This varied 
from course to course. Physics required 10,000 
hours including revision; Russian, 1,800 hours. 
Naval Operations, 400 hours; and Chemistry, 
2,500 hours. It is difficult to give an average amount 
of programming time required since this varies 
according to the complexity of the course. 

Other time 

The other amount of time which was significant 
was that which was necessary to develop graphics 
and filmstrips. Although the actual development 
time was not lengthy, the turnaround time was 2 
months from author to finished product. It was 
necessary, as a result of these difficulties, to restrict 
the use of audiovisuals to an absolute m inim um. 
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Hardware 

The computer system consists of a central proc- 
easing unit and 24 student terminals. Each terminal 
contains a cathode-ray tube (CRT), an image 
projector for filmstrip, an audio source, especially 
for the Russian course, and a typewriter. The 
cathode-ray tube presents the majority of the mate- 
rial and an attached light pen or keyboard is used 
by the student to respond to it. The majority of the 
typewriters were later removed since they were 
used very little and the expense involved in keep- 
ing them was un justifiable. 

The Naval Academy reports few equipment fail- 
ures which were not solved quickly. The computer 
reliability is estimated at 96-98 percent. 

Recommendations 

^Tlie following ^^commendations emanate from 
our TJ,S. Naval Academy experience and under^ 
score the suggestions of many other OAI 
investigators 

1. Before any course materials are considered 
or generated, plan each phase of the project. 
Devote extensive preliminary effort to spec- 
ifying project goals and student-instructor 
goals, to mastering system-capabilities, to 
understanding and appreciating principles 
of sound pedagogy and precepts of educa- 
tional technology, and to determining that 
a systems approach to learning-teaching is 
necessary, practicable, and attainable in 
this circumstance. 

2. Stress and adher© to the reqmrement to 
produce realistic quantities of high quality, 
that is, effective materials. Realize that 
creation of packages of excellent CAI ma- 
terials can no more be produced under 
deadline production methods than new 
ideas can be forcibly formulated in stipu- 
lated numbers of minutes. 

S. Appreciate the fact that the computer is a 
tool, an nstrument which is to serve the 
needs of students, subject matter, and 
teacher, rather than vice versa. It should 
always be subordinate to and supportive of 
the instructional message. Realize that 
.Sophisticated students are unimpressed 



with the gimmickry of blinking lights, 
roll-on messages, caricatures, and even with 
the involvement of personally responding 
to each question when confronted with in 
©fractional messages that are irrelevant, 
confusing, incomplete, unehallenging, im= 
personal, et cetera. 

4. Ensure that OAI materials receive 
thorough validation, that is, analysis of 
content and student performance data to 
ensure that they are both teaching and 
testing as designed to do, and that revisions 
of course materials are based on as clearly 
objective need as possible to attain. 31 

Conclusions 

The following conclusions do not imply any 
criticism of the Naval Academy; they are men- 
tioned only so that Federal EDO’s who are con- 
sidering using OAI can avoid similar problems. 
These problems are not unique to the U.S. Naval 
Academy and would probably occur in most simi- 
lar situations. The Naval Academy has derived 
some of these conclusions through their experience 
with CAI; others have been drawn from similar 
studies. 

1. The time and personnel requirements musr 
be determined before the CAI commitment 
is made. If adequate resources are not avail- 
able, the commitment should not be made. 

2. The project should be started on a small 
scale so that experience can be gained be- 
fore making a large-scale commitment. 

S. It is desirable to begin with a course on an 
already existing computer, 

4, A team of individuals must be available for 
course development, 

5, If sufficient time and personnel have been 
allocated, mediocre or haltgood courses 
should not be accepted, 

6, The courses should be developed in such a 
way that they use the capabilities of the 
computer to the fullest, that is, individ- 
ualize infraction to the maximum. 

7, It is essential that a pilot study of the course 
be conducted before the course is used. 



SUMMARY 



CAI is an instructional method which has the 
capability of highly individualizing instruction. 
But it is only one instructional method among 
many and not appropriate for all learning situa= 
tions and all types of students. If CAI seems to 
be the one which will best fit an agency’s needs, it 



must be determined whether or not the resources 
and personnel are available. The decision to use 
CAI must be made carefully sires the commitment 
of time, funds, and personnel is so heavy and 
would be a great loss if the prog^m were not 
successful 



CHECKLIST OF CAI REQUIREMENTS 



Two checklists follow : the first covers elements 
essential to begin a CAI program, and without 
which no OAI course, regardless of how well it 
is done, can succeed. These items are necessary but 
not sufficient for success. The second, covers ele- 
ments which are essential to the course and along 
with the first group, are sufficient for success.* 

ELEMENTS NECESSARY TO BEGIN A 
CAI PROJECT 

1* Do you have the proper resources for a long- 
term (i.e., 5 year) project? 

A, Funds 

Computer ($6,500 month rental for 
IBM 1800 computer and peripherals 
with 20 percent educational dis- 
count) , 

Terminals ($7,500 month rental for 25 
terminals) . 

Salaries 
et cetera 

B, Personnel 

Authors 
A-Y specialist 
Instructional programmer 
Behavioral scientist 
Programmers 
et cetera 
O, Time 

Author (45- 850 hours per instructional 
hour) . 

Programmer (40-555 hours per instruc- 
tional hour) . 

*The figures cited in these checklists were obtained from 
U.S. Naval Academy reports. 

if 



2. Do you have a subject matter which is relatively 
stable (i.e,, 3 years) which compares with the 
life of a textbook? 

3, Do you have a large number of trainees who 
will be using the course (i.e,, more than a one- 
shot course ? ) 

4. Is this particular subject matter particularly 
suited to CAI (i,e,, well structured, somewhat 
complex ? 

5, Do you plan on using the computer 24 hours a 
day ? If not for instruction, then for some other 
function suc h as administration ? 

ELEMENTS NECESSARY FOE COURSE 

1, Do you have the appropriate blocks (4 hours) 
of time available fo* 1 the authors ? 

2, Do your authors have extensive teaching ex- 
perience, subject matter knowledge, and an ac- 
ceptance of educational technology philosophy* ? 

3, Has the course been validated, tried out on a 
test group before being given for credit ? 

4, Do you have well- written bebaviorally stated 
objectives ? 

5, Do you have precise criterion tests ? 

6, Do you have an efficient method of developing 
graphics ? 

7, Do you have several authors, each with a dif- 
ferent teaching style, for each course to provide 
for the different learning styles of students ? 
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